Classification of Rail Track Defects Based on Computer Vision Using DNN

Abstract

Economic status of the country depends on the Trading which needs transportation. Railways is the most preferred road transportation as most of the profit oriented and movement of people in India is done by elevated railway. Hence it is required to monitor the track health condition frequently using an automated crack detection system. The proposed framework focuses on implementation of python to detect track defects based on Computer vision using image processing techniques. The proposed work uses CNN algorithm through yolov5 model. Yolov5 is one of the best model to achieve highest accuracy in object detection. Yolov5 has become industry standard for object detection due to its speed and accuracy. Here feeding of pre- processed image to CNN classifier to obtain the type of track. The proposed work also helps to identify the severity and non-severity of defects, also suggests the precautions. Automatic communication occur where the message is sent to authorized people of railway department. The Accuracy of the proposed work on the procured images is more than 95%.

Introduction

I. INTRODUCTION

Economic popularity of a country relies upon on road transportation such as railways. The railway is the biggest mode of transportation in India. Hence rail transportation has more significance in India. The modern framework used in Indian Railways interestingly lacks new technologies, consequently probabilities of human error is more. In the past days, rail inspection was manually done, which the exercise of analyzing railway tracks for faulty cracks that may lead to destructive breakdown. According to the US FRA workplace of security analysis track defects are the leading cause of accidents on Railways. The main motive of accidents in railways is accredited to human failure as humans need to go to a specific location and study the cracks and be aware down their location. If he finds negligible cracks, he will forget about which will develop with time and be regarded as most dangerous. Now the present device makes use of the signaling system. The track circuit consists of electrically isolated running rails and a relay, which is used to determine whether the railway track is occupied      or unoccupied. This device is used to detect simple cracks such as broken rails. Even this existing system has drawbacks such as which will not be providing a distinct examination of track cracks such as corrosive track. At the same time the contemporary framework takes greater time for detection.

The proposed framework uses deep neural network approach, where the classification of defective and non-defective images of the rail track are done by neural network classifier. Also it identifies the condition of track and this information is automatically communicated to the authorized people in railway department.

The proposed work identifies five types of railway track defects and a non-defective railway track. The six types of classes are as shown below:

Figure 5 shows the Squats on Railway track which Squats positioned specifically in the low leg of curves, the high leg of shallower curves and in tangent track. Figure 6 shows the Non defective Railway track which is suitable for railway transportation and will not cause any accidents.

The paper is organized as follows: Section II presents the related work with reference to the literature review by showing the previously published work of the railway track crack detection. Section III introduces the proposed work of the system. Section IV presents how  the proposed work is implemented. Section V presents the experimental results and discussion. Section VI concludes the paper and discusses some perspective work.

II. LITERATURE SURVEY

R. Thendral et al. [1] proposal focuses on the detection of railway track cracks routinely using a computer vision- based  technique. The device consists of a rolling digital camera attached beneath a self-moving automobile to seize the photographs and these images considered are the cracked and non-cracked images. The steps in this paper included pre-processing scheme and Gabor transform. In the pre-processing module, the step adaptive histogram adjustment manner is utilized to the pattern photos of railway track to improve contrast in the image, and then additional aspects are extracted from the Gabor magnitude image. These extracted facets are fed as source to deep learning algorithm getting to know neural community classifier to notice the cracks. The accuracy of this machine obtained is 94.9% and the error charge is ;1.5%. Scarlett Liu, [2] introduces the research and contribution of various students in the discipline of visual inspection, summarizes the utility and development of visual inspection technology in the railway industry, and ultimately forecasts the future lookup course of visual inspection technology. Less parameter to get the self assurance price may someday end result in false positive.

Rahman Shafique, [3] ambitions at improving the ordinary railway crack device to tackle the issues with the aid of introducing an computerized railway track fault detection structures using Acoustic analysis. The speedy growth and miniaturization of sensors and electronics tools has made it ubiquitous and on hand on the market.

Mr. Mukul Pande et al. [4] This paper includes a railway track crack detection machine to forestall teach accidents the use of image processing technology. In this paper, Raspberry Pi four B is used to coordinate all the other units in the system. The work in this paper included a Raspberry Pi digital camera which takes the images of railway tracks and sends them to the Raspberry Pi to check for cracks. When the crack is detected region of the crack and area latitude and longitude coordinates are recognized by using the reader module. This paper consists of a Wheel encoder to send the statistics to the mobile about the vicinity of the crack detected.

Rijoy Paul et al. [5] The author of this paper developed an automatic railway track crack detection gadget using photo processing techniques.

This challenge also includes object detection using ultrasonic sensors and photos are captured by means of Raspberry Pi camera. In this paper, Raspberry Pi three is used to coordinate the undertaking of the devices and additionally consists of GPS and WIFI modules to ship the information and place latitude and longitude coordinates where the crack is detected. Erika Steyn [6] proposes Railway upkeep programs global uses rail grinding and rail milling to fight surface defects such as corrugations and rolling fatigue. However, rising applied sciences including of variations on rail grinding and rail milling are gaining pastime in the railway maintenance department. Limited academic literature is available on these alternative surface cure methods. This work includes supporting literature on rail grinding and milling technologies, and gives perception.

Lei Kou [7] discuss the opinions of contemporary lookup and investigation on the defect inspection of rail track in current years. In this paper,  not only used regular ultrasonic and acceleration detection techniques but also includes the  application of computer vision and deep learning for  studying to discover defects on the rail track. Besides, the rising lookup on defect prediction to limit the inspection fees is attractive. Manikandan Rajagopal [8] proposes method that enhances the track photograph the use of adaptive histogram equalization technique and in addition aspects as Grey Level Co- prevalence Matrix (GLCM) and Local Binary Pattern (LBP) function are extracted from the enhanced rail track image. These extracted points are trained and labeled the usage of neural community classifier which classifies the rail song picture into either cracked track or non-cracked track image. S. Nandhini [9] focused on the faults or cracks in rail tracks is imperative in railway management, as it helps to avoid train accidents all through summer time and rainy times. During summer season due to the warmness cracks are fashioned in the tracks, which in the end leads to the slipping of the instruct wheel impacts rail tracks even in wet environments, causing cracks. This system makes use of CNN algorithm to pick out cracks on railway tune surfaces and gain their corresponding masks, which can then be used to extract other inspection applicable properties.

III. PROPOSED METHODOLOGY

A. Hardware Requirement

Wi-Fi module is connected to the LCD and Power supply which help to transfer the data to the cloud over internet as shown in Fig.7. The data will be directed to cloud by means of the Wi-Fi module and data can be viewed.A Transistor Transistor Logic is abbreviated as TTL which is used to interface between Hardware and Software as shown in fig 8. The Txd pin of TTL is connected to Rxd pin of node MCU.

It can show 16 characters per line and there are two such lines. In this LCD every character is displayed in 5*7 Pixel matrix as shown in Fig 9.A power   supply is   an   electrical   device   that supplies electric power to an electrical load as shown in fig 10.

B. Software Requirement

Fig 11 Tabular column of Software requirement

IV. DESIGN AND IMPLEMENTATION

Figure 12 refers to the block diagram of the proposed device where the essential primary objective of the proposed work is to reduce labor work and time consumption in Railways Department while inspecting tracks. Yolov5 is the heart of the system where training of data is done in Yolov5 framework. Here CNN is used for detection of defected track and non defective track. NMS is used with Yolov5 to suppress the non-max output. Significantly the synchronization of Hardware and Software is done using nine single characters.

In the above figure which depicts the Block diagram of Image processing , where the Track defect images are fed as input to the python software, where images are read, Resizing of image takes place in order to get uniform sized images. Conversion of image from gray to binary occur to find the region of interest which is the portion of the image that needs to processed further. Keras filters are used to analyze the percentage of error. The simulated model provides a good accuracy and prediction score.

A. Image processing and annotation

Figure 14 refers to the window where the user is labelling the track images in make sense. Referring to Figure 14 where left part of window displays the uploaded  images, middle part of the window displays the image that needs to be labelled currently, Right part of the window displays  the label name. Figure 15 refers to the robo flow window where 70% of dataset is classified as train, 20% of dataset is classified as valid, and 10% of dataset is classified as test.

B. Implementation of CNN and YOLOv5

The figure 16 shows the flowchart for pre-processing of datasets. The acquired data is converted from Gray to Binary to extract the required region of the image which need to undergo processing. Median filtering technique is used to remove the noise in the image. Global thresholding is used to replace the negative values by 0. The output from previous step is passed through the high pass filter where the image sharpening occurs. In this module emphasis on fine parts of image.

The figure 17 shows the flowchart for the feature extraction. we use a method called Histogram Orientation Gradient (HOG).  It involves multiple steps like finding Gx and Gy, which are gradients about each pixel in the x and y axes. Then Gx and Gy are calculated. After finding the Gradiant x and Gradient y it is required to find the magnitude of angle and find frequency of each angle and plot in histogram.

The figure 18 depicts the classification of image. Convolving the matrix mean the output from the high pass filter is processed by convolution layer to extract the features and it also retains the relationship between pixels of the input image. It is basically the multiplication of two inputs, followed by rectifying the negative values and replace it by 0. The output from previous step is sent to relu layer to reduce the noise and maxpooling layer is used to reduce the matrix size. Finally fully connected layer is used to classify the output.

The figure 19 depicts The architecture of Yolo divides the image into  a grid of S*S size. If the center of the box bounded around the object is in that grid, then this grid is responsible for the  detection of object.    

If the confidence score = 0, then there is no object exists in the grid.

If the confidence score = IOU, then there exists object present in the grid.

Each bounding box contains five parameters such as(x, y, w, h, confidence score) where (x, y) coordinates represent the center of the bounding box. (h, w) coordinates represent the height and width of bounding box. The confidence score represents the presence of an object in the bounding box. Here we multiply conditional class PROand the individual box confidence predictions which gives the class specific confidence score for each box. These scores encode both probability of that class appearing in the box and how well the predicted box fits the object. After applying the non maximal suppression, the final predictions are generated.

  As shown in figure 20, ESP 8266 is a self sufficient MCU which has inbuilt WIFI in it, it has the ability for data processing read and control the GPIO pins. The proposed work uses ESP8266 to send character and message received from software to hardware. The Txd pin of TTL is connected to Rxd pin of ESP8266.TTL(Transistor Transistor Logic is a UART convert module which creates a virtual com port using USB to interface between hardware and software. LCD is a liquid crystal display which has 16x2 dimensions used to display message. Power supply is used to supply power to ESP and Lcd, Cell phone is used to input the video clip and display the received message.

Figure 21 depicts proposed system model .The proposed system model works in such a way that the TTL is connected to the laptop to interface the hardware and software. Power supply is turned on to activate the hardware components . Open project folder in laptop, in the path type cmd in order to open the command window with project folder path address. Type "flask run" command and execute, the IP address of "AI PROCTORING IN INDIAN RAILWAYS" webpage is

obtained which needs to be opened in web browser. Once the web page is opened, more importantly it will allow only the authenticated users. If the user has permission then the user can use the proposed system.

The type of track defect is detected which is displayed on the laptop screen and the same is displayed on the LCD display. Hardware and Software synchronization is done by sending single character for each detected class such as 'H', 'G', 'S', 'T', 'R', 'Q', 'B', 'C', and 'N' for Head check severity, Head check non-severity, Squats severity, Squats non-severity, Rolling fatigue severity, Rolling fatigue non-severity, broken severity, corrosion non-severity, and non-defective respectively. .The detected track defect, condition of track and precautions are sent to the authorized people in Railway department through telegram. Based on the type and condition of track defect the authorized people in Railway department can take action.

D. Flowchart and Working

 When the power supply is ON, Lcd and ESP module gets activated. TTL is to be connected to laptop, and Lcd displays the message "Railway crack monitoring system". Once the system is turned on, there are two possibilities of inputting the image either by selecting the image within the laptop or by showing the video to the PC's camera. As shown in flowchart if the confidence score is greater than 0.45 and IOU is greater than 0.25 then the type of crack is detected and is shown on lcd screen. The detected type of crack is compared against the character which is to be sent to hardware such as 'H', 'G', 'S', 'T', 'R', 'Q', 'B', 'C', and 'N' for Head check severity, Head check non-severity, Squats severity, Squats non-severity, Rolling fatigue severity, Rolling fatigue non-severity, broken severity, corrosion non-severity, and non-defective respectively. These characters synchronize hardware and software.

V. RESULTS AND OBSERVATIONS

Figures shows the detection of defective rails and is also displayed on LCD display. Simultaneously data such as type of defect with the precautions and condition of defect is sent through telegram.

Conclusion

Through this method of using Deep Learning forthe detection of cracks in railway we are able to eliminate the cost of using expensive audio based or physical method of finding cracks. We have achieve an accuracy of more than 95% after testing it with the validation dataset. If this system is brought in railways, the accident could be controlled and the same message is sent automatically to the control room and since it’s a completely automated system it can be used in village areas by which man power is reduced and time is saved. Moreover the system minimize the train accidents caused by train track cracks . There by possible to save precious lives of passengers and loss of economy. It also saves the time and money for identification of track defects. The future of railway inspection include neural network analysis to improve defect detection. So the proposed system can be implemented along with self driving vehicle with a high resolution camera where the camera takes the images of rails and fed them automatically to the deep learning neural network algorithm for the detection of cracks. Simultaneously an alert message is sent to the Control room through hardware attached to the proposed system and in future GPS can be added to find the location of the cracks.

References

[1] R. Thendral, A. Ranjeeth “Computer Vision System for Railway Track Crack Detection using Deep Learning Neural Network”, DOI: 10.1109/ICSPC51351.2021.9451771 [2] Rahman Shafique, Hafeez-Ur-Rehman, Furqan Rustam “A Novel Approach to Railway Track Faults Detection Using Acoustic Analysis” in 2021, DOI: 10.3390/s21186221 [3] Arun Kumar R, Vani Shree K, Shweta K, Nandini C, Shweta G “Railway track crack detection” IJERT, ISSN: 2278-0181, Volume 8, Issue 15 [4] Scarlett Liu, Quandong Wang, and Yiping Luo “A review of applications of visual inspection technology based on image processing in the railway industry” Transportation Safety and Environment, 2020, Vol. 1, No. 3 186–205 doi: 10.1093/tse/tdz007 [5] Ms. Mini Magdline, Mr. Mukul Pande, “Accident prevention & crack detection system for railway tracks” 2021 IJCRT | Volume 9, Issue 4 April 2021 | ISSN: 2320-2882 [6] Aliza Raza Rizvi, Rauf Khan “Crack Detection in Railway Track Using Image Processing”, Corpus ID: 64931603 [7] Erika, Steyn “Rail Surface treatment – Grinding and milling – Competing or complementary technologies” in 2019 https://hdl.handle.net/20.500.12380/300204 [8] Lei Kou, “A Review of Research on Detection and Evaluation of Rail Surface Defects” 2021, Institute of Railway Systems and Public Transport, TU-Dresden, Reitbahnstr.35, 01069, Dresden, Germany [9] Manikandan Rajagopal, M. Balasubramanian, S. Palanivel “An Efficient Framework to Detect Cracks in Rail Tracks Using Neural Network Classifier” ISSN 2007-9737, Vol. 22, No. 3, 2018, pp. 943–952 doi: 10.13053/CyS-22-3-3024 [10] S. Nandhini, Mohammed Saif, Kailash Venthan P, Sridhar E, “Robust Automatic Railway Track Crack Detection using Unsupervised Multi-Scale CNN” IJARSCT Volume 12, Issue 3, December 2021

Copyright

Copyright © 2022 Spoorthi P A, Anitha G S, Bhavana S J, Jayashree A M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.